Towards an Unsupervised Method for Network Anomaly Detection in Large Datasets

In this paper, we present an e↵ective tree based subspace clustering technique (TreeCLUS) for finding clusters in network intrusion data and for detecting known as well as unknown attacks without using any labelled tra c or signatures or training. To establish its e↵ectiveness in finding appropriate number of clusters, we perform a cluster stability analysis. We also introduce an e↵ective cluster labelling technique (CLUSLab) to label each cluster based on the stable cluster set obtained from TreeCLUS. CLUSLab is a multi-objective technique that employs an ensemble approach for labelling each stable cluster generated by TreeCLUS to achieve high detection rate. We also introduce an e↵ective unsupervised feature clustering technique to identify the dominating feature set from each cluster. We evaluate the performance of both TreeCLUS and CLUSLab in terms of several real world intrusion datasets to identify known as well as unknown attacks and find that results are excellent. 2 M. H. Bhuyan, D. K. Bhattacharyya, J. K. Kalita